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Abstract 

This article reviews the literature on multi-sensory technology and, in 
particular, looks at answering the question: ‘What multi-sensory’ 
technologies are available to use in a science, technology, engineering, arts 
and mathematics (STEAM) classroom, and do they affect student engagement 
and learning outcomes? ’ Here engagement is defined as motivation, interest, 
curiosity and attitude. This review identifies tools and software from a 
burgeoning, yet limited literature. To answer this question, this review will 
include the following: defining learning styles; background on multi-sensory 
instruction; multi-sensoiy education today; STEM and the rise of STEAM; a 
look at some multi-sensory tools; and the benefits of using multi-sensoiy 
technologies in education. 
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Introduction 

Concern was raised by the United States National Academies in 2006 about the declining state of 
STEM (Science, Technology Engineering and Mathematics) education in the US. In response, 
there has been an increased focus on these disciplines including a variant form known as STEAM, 
that is, Science, Technology, Engineering, Arts and Mathematics. 

At the same time, the information technology industry has seen a growing interest in the creation 
of new technologies for education, entertainment and more. Jayakanthan (2002) suggested that the 
computer game industry, which includes games for educational purposes, has become larger than 
the film and music industries. Multi-sensory technologies, in formats such as virtual reality, 
simulations, and virtual field trips (Raskind, Smedley, & Higgins, 2005), and tablet computers 
have been developed and have found a ready home in STEAM education. This paper is concerned 
with these technologies and their emerging role in STEAM education. 

This paper will review the literature on Science, Technology, Engineering, and Mathematics 
(STEM) and the rise to STEAM to better understand the importance of these subjects. It will also 
identify the multi-sensory technologies with particular potential to support STEM/STEAM and to 
affect student engagement and student learning outcomes. 
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Defining learning styles 

We all interact with the world around us with our five senses, but we process the information 
received in our distinct ways (Taljaard, 2016). These different processing methods have been 
categorised into what educators call “preferred learning styles” (Halpern, 2003; Wehrwein, Lujan, 
& Dicarlo, 2007). Learning styles can be described as “cognitive, affective, and physiological 
traits that are relatively stable indicators of how learners perceive, interact with, and respond to the 
learning environment” (Keefe, 1979, p. 4). It has been contended that people have different and 
distinct sensory style preferences (Fleming & Baume, 2006; Halpern, 2003) and further suggested 
that “our preferences are part of who we are” (Fleming & Baume, 2006, p. 4). A student who may 
find clarity and motivation in imagery may, therefore, migh keep using this strategy to improve 
his/her learning outcomes. Some students are active listeners and can understand by listening 
while others may prefer field trips or more active engagement (Fleming, 2001). 

According to the VARK model, there are four learning styles - visual, auditory, read-write and 
kinaesthetic (Fleming, 2001). The VARK model is a questionnaire originally designed for business 
but provided free to the education sector which helps students by suggesting the strategies they 
should be using. It was initially developed in 1987 by Neil Fleming, and Colleen Mills (1992) after 
Fleming observed how effective teachers did not reach some learners. Some of Fleming and 
Baum’s (2006) ideas include: 

• modal preferences influence individual’s behavior and learning; 

• information accessed using strategies that are aligned with a student’s learning preferences, is 
more likely to be understood and be motivating; and, 

• the use of strategies aligned with a student’s learning preference is likely to lead to a deeper 
approach to learning, and active and effective metacognition. 


(Fleming & Baume, 2006, p. 4) 

VARK relates to how we acquire new knowledge and does not involve intelligence. It can 
therefore be said that the VARK model allows for all individual learners to attain knowledge in an 
interesting and captivating environment in a way that suits their preferred learning style. VARK 
has resulted in the development of various multi-sensory learning strategies, but multi-sensory 
strategies and techniques are not new and have been used for many years. 

Background on multi-sensory instruction 

The first recorded use of multi-sensory techniques was in 1920. Dr Samuel Torrey Orton ran a 
mobile health clinic in Iowa, United States of America, where he dismissed the common idea that 
dyslexia was caused by vision problems and proposed that it was rather a “specific reading 
disability” (Orton, 1925, p. 1095). Orton (1925) further suggested that the tendency of reversing 
letters in dyslexia could be corrected by kinaesthctic-tactile reinforcement of visual and auditory 
associations. Orton was greatly influenced by the work of Grace Fernald, who created a 
kinaesthetic method of tracing words, known as “writing in the air,” while saying the names of 
sounds out aloud (Hallahan & Mercer, 2001). 

Cleland and Clark (1966) later proposed a theory of the positive effects of multi-sensory 
environments. They suggested that stimulating the senses of individuals with cognitive 
impairments through smell, sound, touch and sight can improve development, communication and 
behaviour. 

Rains, Kelly, and Durham, 2008) contended that the theories of Piaget and Vygotsky show the 
importance of employing multiple methods in education. This is because students of a similar age 
group are not always at the same level of mental readiness. 
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Since 2006, and due to advances in technologies, there has been a growing interest in the field of 
multi-sensory technologies in education and research continues through projects like the Newton 
Project (http://www.newtonproject.eu), which is funded by the European Union’s Horizon 2020 
Research and Innovation programme. 


Multi-sensory education 

Multi-sensory education can be explained as an instructional method which uses visual, auditory, 
kinaesthetic and tactile ways to educate students (Joshi, Dahlgren, & Boulware-Gooden, 2002). It 
involves teaching through hearing and speaking, seeing and perceiving, and touch, movement and 
action (Taljaard, 2016). Multi-sensory teaching techniques stimulate learning by engaging students 
on multiple levels. They encourage students to: 

• Gather information about a task 

• Link information to ideas they already know and understand 

• Perceive the logic involved in solving problems 

• Learn problem solving tasks 

• Tap into nonverbal reasoning skills 

• Understand relationships between concepts 

• Store information and store it for later recall 

(Clifford, n.d., p. 2) 

It has also been demonstrated that the human brain learns and functions optimally in situations in 
which information is amalgamated within multiple sensory modalities (Shams & Seitz, 2008). It 
can, therefore, be argued that multi-sensory protocols are more efficient for learning as they are 
closer to natural settings than unisensory protocols. In line with research, a current tendency exists 
to simulate deep multi-sensory learning through doing mimicking how young children study their 
surroundings using all their senses (West, 1994). Prigge (2002) proposed that a “solid” experience 
is one of the best ways to make lasting neural connections through a more direct interaction with 
the surroundings. This is because the brain processes 3D images in significantly different ways to 
how it processes 2D images. It has also been contended that introducing a new sense may open 
new neural pathways (Liu & Chiang, 2014). 

Traditional learners are viewed as either low- or non-tactual as they are mostly auditory or visual 
students and learn the way most teachers teach. These students are better represented among 
females than males (Dunn & Dunn, 2005) and, as suggested by Honigsfeld and Dunn (2003), 
males tend to be strongly tactual and kinaesthetic. A further group of students are low auditory 
and/or low visual and learn best with kinaesthetic strategies (Bauer, 1991). The importance of 
haptic/tactile experiences has also been demonstrated by basic psychological studies (Jakesch, 
Zachhuber, Leder, Spingler, & Carbon, 2011). 

Praveen (2016) contended that “if a child is not learning in the way you teach, change your 
teaching strategy and teach the child in the way he learns!” (para. 1). A challenge, therefore, exists 
for designers of learning materials to engage as many sensory channels as possible so that students 
can be accommodated in the various ways in which they learn. Gardner (1983), a Harvard 
developmental psychologist, suggested pluralising learning, a strategy which involves presenting 
topics in a variety of ways so as to reach more students. This also conveys what understanding 
something well means. Students need to become adept in understanding how they learn best and 
the skills required to learn in areas they do not like or areas they do not believe would have any 
future benefit; in this way, they can take responsibility for their learning (Kolb & Kolb, 2012; 
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Prigge, 2002). Ultimately, learning styles are divided into four senses - visual, aural, reading, and 
kinaesthetic (Fleming, 2001; Walling, 2014). Learning style theories, despite practical use in many 
classrooms, are still very much contested (Walling, 2014). But as a theory, they make sense to 
teachers and learners alike. 

Complementary information, used to develop multi-sensory interfaces, provides universal access 
through combinations of sound maps, haptic interfaces and speech (Jacobson, 2004). Although in 
Jacobson's (2004) research, the focus is on the visually impaired, it can be said that the findings 
could also be applied more generally. For example, maps presented in a multimodal form have the 
potential to provide a source for learning about the environment and offering opportunities for 
further investigation. 


STEM and the rise of STEAM 

STEM has been identified as a significant national reform in education and curriculum in the 
United States of America, to prepare students for the global economy of the 21st century. 

Generous support has been allocated to education, in the belief that the US is becoming less 
competitive in STEM fields compared to Asian countries (Piro, 2010). 

A study by Gaugin (2008) showed that people involved in the Arts represent a large part of the 
USs workforce. Further research has demonstrated that contribution to the arts leads to calculable 
cognitive gains (Preminger, 2012). It has also been shown that training in a musical instrument 
enhances verbal ability and non-verbal reasoning (Piro, 2010). As a result, there has been a change 
in education globally, to include the arts in STEM subjects. 

STEAM (sometimes written as STE@M (Yakman, 2008)) has become a model of how boundaries 
between educational subjects can be removed. STEAM revolves around the idea that it is the 
“interpretation of Science and Technology through Engineering and the Arts, all based in 
Mathematical elements” (Kwon, Nam, & Lee, 2011, p. 783). STEAM also allows for these 
subjects to be delivered in an engaging and deeply embedded way (Yakman & Lee, 2012). 

It can, however, be said that for STEAM to be successful, there has to be an understanding that it 
not simply about adding the Arts to the list of Science, Technology, Engineering and Mathematics. 
It is also about looking for associated standards between these STEM subjects and the 
implementation of those standards into the curriculum. It can also be suggested that STEAM is 
ideally suited for the incorporation of multi-sensory tools into the classroom. 


Multi-sensory tools 

The multi-sensory tools reviewed in this section are: virtual reality, tablet computers, JAWS and 
SALS. 


Virtual reality 

Virtual reality can be defined as a text- and graphics-based environment that is simulated by a 
computer (Auld & Pantelidis, 1994). Today, however, virtual reality can mean so much more, 
from simple simulation programs delivered via a mobile phone to full immersion using specialised 
equipment like Oculus Rift, a virtual reality headset for 3d gaming. No matter the complexity, the 
user uses a screen to display the virtual world and has the capability to interact with this world 
with the help of a mouse, joystick or gyroscope as well as voice command input, hand motion 
input or even self-motion, which makes virtual reality a great multi-sensory tool to use in 
education. 

Heeter (1992) suggested that, in a virtual reality situation, human senses are no longer part of an 
exhaustive list as other new senses can be added. Some games include tactile stimuli from 
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touching buttons and joysticks in the virtual cockpit. It can be said that it feels like driving in a real 
cockpit. In the same game, you are also not limited to one set of eyes. A primary monaural eye is 
always visible with secondary eyes that can be up to 50 feet (—15 metres) above your head. Players 
surveyed suggested that, when they touched animated objects in these worlds, their physical and 
emotional responses helped to convince them that the experience was real (Heater, 1992). 

There has been a steady increase in the development of educational software. Some involves full 
immersion in virtual environments while others limit confusing background information and 
accentuate specific sensory information (Raskind et ah, 2005). This also allows students with 
cognitive disabilities to learn complex concepts with greater physical and emotional safety and 
students with physical disabilities to compete in sports events (Powers & Darrow, 1994). 

Immersion is one of the key features that virtual reality adds to modelling as a means of 
constructivist learning. It allows learners to develop the impression that they are taking part in a 
world that is realistic enough to make them believe it is true (Heeter, 1992). Immersion may also 
make concepts more memorable and thereby allow students to build better mental models. Another 
feature is that it keeps the student’s attention focused on the virtual environment. When using a 
head-mounted display, students are not subjected to the same distractions as in traditional learning 
environments. 

A multi-sensory plus for virtual reality headsets is virtual field trips. As part of the recent push for 
STEAM subjects in education, Google created Google Expeditions. Optimally, these virtual field 
trips, in conjunction with other learning methods, provide a solid instructional method (Bellan & 
Scheurman, 1998). 

Virtual reality and, more particularly, virtual field trips, have advantages and disadvantages. In 
general, the disadvantages are caused by inadequate preparation or incorrect use. To avoid these, 
teachers should make sure that students are aware of how to use the tools and what is expected of 
them. A follow-up after the virtual field trip is also suggested (Bellan & Scheurman, 1998). The 
advantages of virtual field trips make it all worthwhile. Students can experience sensory overload 
on actual field trips. Virtual field trips allow students to focus their sensory input when on the 
actual trip, but knowing in advance what to expect. But even better, these virtual trips allow 
students to experience places that may be too dangerous to visit, like to the top of a volcano, or 
places they may not be able to afford to visit, like far away countries. 


Tablet computers 

Tablet computers allow students to document their learning by using multi-sensory functionality 
(Reich, 2013) and come equipped with the functionality to meet the needs of students and can 
facilitate learning in the student’s preferred learning style (Walling, 2014). 

The features of tablet computers allow students to watch educational videos, record their lab 
experiments/progress with the built-in camera, develop multimedia presentations and engage in 
many preferred learning styles. Walling (2014) argued that tablet computers are not merely high- 
tech toys but, rather, are toolboxes for learner engagement. Walling (2014) further suggested that 
the transition to using tablet computers in education is a natural process for teenagers as they are 
very similar to smartphones. 

Some of the features noted by Walling (2014) include: the voiceover feature for spoken 
descriptions of everything on the screen; Siri, the personal assistant activated by voice prompts; 
dictation to convert speech to text; and Braille displays and keyboards. Other features include 
closed captions, assistive touch, and the gyroscope to record the movement of the device. 

It can, however, be said that tablet computers and multimedia applications have their inherent 
limits when it comes to multi-sensory learning as they are limited to only a couple of senses. The 
inclusion of add-ons like apps or tactile screen covers allow more sensory interaction. 
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JAWS and SALS 

Two other, lesser-known tools with limited research are JAWS and SALS. 

• JA WS is a text-to-speech screen reader that allows learners with low vision to undertake 
online research, thereby using their auditory sense in the classroom or at home. Compared to 
other software, it provides extensive audible functionality, even including real-time probe 
readings. In a science classroom, this will allow students who are visually impaired to install 
sensors and probes and even access and manipulate data (Supalo, Wohlers, & Humphrey, 
2011 ). 

• SALS is the acronym for Submersible Audible Light Sensor, a glass wand with an embedded 
light sensor. This allows the measuring of colour intensity changes. The intensity changes are 
then converted into audible tones (Supalo et ah, 2011). 

The above tools were used in a science camp for students who are visually impaired (Supalo et ah, 

2011), but it could be suggested that these tools can also be included/adapted for general inclusion 

in multi-sensory education. 


What are the benefits of using multi-sensory technology in 
education? 

The reported benefits of using multi-sensory technologies in education are student engagement and 
improved learning outcomes. 


Student engagement 

Evidence of heightened student engagement through the use of technologies have been noted as 

follows: 

• Students in a bookmaking course were found to be more motivated when using guided 
imagery and writing warm-ups (Peterson-Stroz, 1997). 

• Similarly, some multimedia applications and multimedia tools were found to increase 
students’ motivation to learn. They also allow for a better understanding of the subject matter 
(Philpot, Hall, Hubing, & Flori, 2005). 

• When it comes to virtual reality, research has shown that students are captivated by these 
immersive worlds. As a result, they devote more time and concentration to the task at hand 
(Wexelblat, 1993). 


Learning outcomes 

A number of researcher over time have investigated the impact of technologies on learning 

outcomes. These include: 

• Imagery and visual perception were found to be especially influential (Borst & Kosslyn, 

2010). Jampole, Konopak, Readence, and Moser (1991) found that the use of visual sense 
could enhance the quality of writing when examining the effects of guided imagery on 126 
students. 

• Language students learning by using multi-sensory experiences were found to have a better 
understanding of new knowledge (Pohan & Kelley, 2004). 

• Shams and Seitz (2008) suggested that multi-sensory techniques in education can better mimic 
real-world settings and are therefore more effective for learning. 
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• Tong and Kong (2011) suggested that one can make a piece of writing meaningful and 

interesting using sensory experiences. It can therefore also be suggested that STEAM subjects 
can also be found to be more interesting and meaningful through multi-sensory learning. 

Further research is needed to determine the efficacy of these approaches and tools on engagement 
and learning outcomes. 


Conclusion 

There is an apparent gap in research when it comes to technology-enhanced multi-sensory 
education, and where research does exist, it is limited to its use in learning how to read or with 
reference to assisted devices for the visually impaired. Most available research focuses on dyslexia 
and a manual approach to multi-sensory education. 

STEAM subjects have become a major focus for educators in many countries, developing from the 
initial STEM subjects. It can be said that educators have come to realise the importance of the Arts 
and that it is not just about adding Art as a subject, but also about looking for associated standards 
between these STEM subjects and their implementation into the curriculum. 

The technologies reviewed here allow for better engagement and increased learning outcomes as 
they allow students to learn in their preferred learning styles. They also make learning fun and 
allow students to connect to real-life situations. 

It can also be suggested that by taking into account the current shortcomings of technology- 
enhanced tools, the multi-sensory learning tools of the future may promote even higher 
engagement and increase student learning outcomes. 

Dewey (1916) argued that the problems with motivation and transfer of learning are caused by the 
disharmony between the textbooks and learners’ life experiences. These problems can be 
overcome with the use of multi-sensory technology in education. This paper agrees with the 
contention that “perhaps the most important single proposition that the educator can derive from 
Piaget’s work and its use in the classroom is that children, especially young ones, learn best from 
concrete activities” (Herbert & Opper, 1969, p. 221). 

This paper has presented a review of some of the multi-sensory activities which could be used to 
enhance learning in STEM/STEAM education. It also considers the impact of technology use on 
student learning. There is much still to be learned as technologies continue to develop and make 
greater use of human senses. 
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